Learning curves from a modified VC-formalism: a case study

In this paper we present a case study of a 1-dimensional higher order neuron using a statistical approach to learning theory which incorporates some information on the distribution on the sample space and can be viewed as a modification of the Vapnik-Chervonenkis formalism (VC-formalism). We concentrate on learning curves defined as averages of the worst generalization error of binary hypothesis consistent with the target on training samples as a function of the training sample size. The true learning curve is derived and compared against estimates from the classical formalism and its modification. It is shown that the modified VC-formalism improves the VC-learning curve by a factor of and also produces a meaningful result for small training sample sizes where VC-bounds are void.